United States Patent im 

Maturi et al. 



ininiiiiiiiiiii 



US005731850A 
[ii] Patent Number: 
[45] Date of Patent: 



5,731,850 
Mar. 24, 1998 



[54] HYBRID HIERARCHL4L/FUIX-SEARCH 
MPEG ENCODER MOTION ESTIMATION 

[76] Inventors: Gregory V. Maturi, 2062 Paseo del 
Oro. San Jose, Calif. 95124; Vlvek 
Bhargava, 1508 Fujiko Dr.. San Jose, 
Calif. 95131; Sho Long Chen, 14411 
Quito Rd, Saratoga, Calif. 95070; 
Ren- Yah Wang, 1121 Buckingham Dr.. 
Los Altos, Calif. 94024 

[21] AppL No.: 485,030 
[22] Filed: Jun. 7, 1995 

[51] Int CI 6 H04N 7/36; H04N 7/50 

[52] U.S. CI « 348/699; 348/416 

[58] field of Search 348/416. 699; 

H04N 7/36, 7/50 

[56] References Cited 

U.S. PATENT DOCUMENTS 

5,134,478 7/1992 Golin 348/415 

5,144,424 9/1992 Savatier „ 348/405 

5,144,429 9/1992 Haghirietal .... 348/416 

5,193,004 3/1993 Wang et al - 348/413 

5,200,820 4/1993 Gharavi „„ 348/416 

5,247,363 9/1993 Sunetal 348/616 

5301.019 4/1994 Gtta 348/416 

5,889,108 8/1994 Coleman etal. 348/408 

OTHER PUBLICATIONS 

EDDY. Chris. "The MPEG Compression Algorithm". PC 
Gmphics <* Video. Jul. 1994, pp. 52-55. 
MPEG Software Simulation Group. "MPEG-2 Encoder/ 
Decoder* 1 Version 1.1, Jun. 1994. MPEG-L@netcom.com, 



MPEG Software Simulation Group. "Makefile for 
mpeg2encode. M 1994. MreG-L@netcom.com. 
"ISO/IEC 13818-2:1995(E) n Recommendation 
rrU-TH.262 (1995E). 1995. 

•TSO/IEC/JTCI/SC29/WG11 Coded Representation of Pic- 
ture and Audio Information. Test Model 5 W Document 
AVC-491. Version 1. Apr. 1993. 

Primary Examiner — Howard W. Britton 
Attorney, Agent, or Firm— Donald E. Schrciber 



[57] 



ABSTRACT 



An apparatus and method for determining inter-frame 
motion during compression of digital video data incorpo- 
rates a computationally efficient hierarchical block- 
matching motion estimation technique in conjunction with a 
full-search block-matching approach. In the hierarchical 
block-matching method, a macrobiock is filtered and 
decimated, and a search area is also filtered and decimated. 
A block-matching search is performed within the filtered and 
decimated search area. An augmented block in the original 
search area that corresponds to the block in the decimated 
search area that provided the best match with the decimated 
macrobiock is then compared with the original macrobiock 
to determine a motion vector. Operating parameters specify 
the search range based on the type of frame being processed, 
i.e. P-frame or B-frame. and, in the case of B -frames, the 
distance of the B-frame from the reference frame. If the 
operating parameters specify a search range above an estab- 
lished threshold, the hierarchical block-matching search is 
performed. If the operating parameters specify a search 
range equal to or below the established threshold, the 
full-search block-matching search is performed. 

17 Claims, 6 Drawing Sheets 
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HYBRID HIERARCHIAIVFULL-SEARCH Refening to FIG. 1, MPEG-1 and MPEG-2 each divides 

MPEG ENCODER MOTION ESTIMATION a video input signal generally a successive occurrence of 

frames, into sequences or groups of frames ("GOF") 10, also 

RAncORO! IND OF THE INVENTION refOTed t0 35 a ° f P ictUFeS H 301 ^ Thc 6™* in 

BACKORUUND OF lHblNViiNUUN ^ respective GOFs 10 are encoded into a specific format 

1. Field of the Invention Respective frames of encoded data are divided into slices 12 

The present invention pertains to video image compres- representing, for example, sixteen image lines 14. Each slice 

sion. Specifically, the present invention pertains to an appa- 12 is divided into macroblocks 16 each of which represents, 

ratus and method for determining the inter-frame movement for example, a 16x 16 matrix of pels. Each macroblock 16 is 

of macroblocks. 10 divided into 6 blocks including four blocks 18 relating to 

2 Description of the Prior Art luminance data and two blocks 20 relating to chrominance 

MPEG is a video signal compression standard, estab- data. The MPEG-2 ^ocol encodes luminance and chrorai- 

lished by the Moving Picture Experts Group ("MPEG") of ^ separately and men^nibines the encoded video 

Jr r\ * ' to* j VvL • llDCP . data into a compressed video stream. The luminance blocks 

the International Standardization Organization MPEG is a ^ ^ 8*8 matrices of pels 21. Each chromi- 

mulUstage algorithm thai .integrates a number ofweUknown 15 oancc Wock an g x g of ^ Kbiting to mc 

data compression techniques into a single system. These emire 16xl6 matrix of peU ^ represented by the macroblock 

include motion-compensated predictive coding, discrete 16 A£ta mc vidco ^ ^ cnco ded {% [ s then compressed, 

cosine transform ("DOT), adaptive quantization, and van- buffered, modulated and finally transmitted to a decoder in 

able length coding ("VLC"). The main objective of MPEG accordance with the MPEG protocol. The MPEG protocol 

is to remove redundancy which normally exists in the spatial 20 typically includes a plurality of layers each with respective 

domain (within a frame of video) as well as in the temporal header information. Nominally each header includes a start 

domain (frame- to- frame), while allowing inter-frame com- code, data related to the respective layer and provisions for 

pression and interleaved audio. An MPEG-1 decoder is adding header information. 

specified in ISO Recommendation ITU-T H.262 (1995E), There are generally three different encoding formats 
dated January 1995. A prototype MPEG-2 encoder is speci- 25 which may be applied to video data. Intra-frame coding 
fied in the ISO document 4 Test Model 5", Document AVC- produces an T block, designating a block of data where the 
491, Version 1, dated April. 1993, and a prototype software encoding relies solely on information within a video frame 
MPEG-2 encoder is published by the MPEG Software where the macroblock 16 of data is located. Inter-frame 
Simulation Group. The preceding ISO publications and the coding may produce either a "F* block or a block. A "P" 
prototype software MPEG-2 encoder are hereby incorpo- 30 block designates a block of data where the encoding relies 
rated by reference. on a prediction based upon blocks of information found in 
There are two basic forms of video signals: an interlaced a prior video frame. A "B* block is a block of data where the 
scan signal and a non-interlaced scan signal An interlaced encoding relies on a prediction based upon blocks of data 
scan signal is a technique employed in television systems in from surrounding video frames, i.e., a prior I or P frame 
which every television frame consists of two fields referred 35 and/or a subsequent P frame of video data, 
to as an odd-field and an even-field. Each field scans the One means used to eliminate frame-to-frame redundancy 
entire picture from side to side and top to bottom. However, is to estimate the displacement of moving objects in the 
the horizontal scan lines of one (e.g., odd) field are posi- video images, and encode motion vectors representing such 
tioned half way between the horizontal scan lines of thc motion from frame to frame. The accuracy of such motion 
other (e.g., even) field. Interlaced scan signals are typically 40 estimation affects the coding performance and the quality of 
used in broadcast television ("TV") and high definition the output video. Motion estimation performed on a pel-by- 
television ("HDTV"). Non-interlaced scan signals are typi- pel basis has the potential for providing the highest quality 
cally used in computer systems and when compressed have video output, but comes at a high cost in terms of compu- 
data rates up to 1.8 Mb/sec for combined video and audio. tatlonal resources. Motion estimation can be performed on a 
The Moving Picture Experts Group has established an 45 block-by-block basis to provide satisfactory video quality 
MPEG-1 protocol intended for use in compressing/ with a significantly reduced requirement for computational 
decompressing non-interlaced video signals, and an performance. 

MPEG-2 protocol intended for use in compressing/ U.S. Pat. No, 5*200,820, entitled "Block-Matching 

decompressing interlaced TV and HDTV signals. Motion Estimator for Video Coder" that issued Apr. 6, 1993 

Before a conventional video signal may be compressed in 50 on an application filed Apr. 26, 1991 in the name of Hamid 

accordance with either MPEG protocol it must first be Gharavi ("the Gharavi patent* 1 ), discloses a method and 

digitized. The digitization process produces digital video apparatus for performing block-matching motion estimation 

data which specifies the intensity and color of the video in a video coder which estimates the motion vector associ- 

image at specific locations in the video image that are ated with each block of pels in a current coding frame. In the 

referred to as pels. Each pel is associated with a coordinate 55 method disclosed in the Gharavi patent referred to herein as 

positioned among an array of coordinates arranged in ver- a "full- search" method, the motion vector for each block in 

deal columns and horizontal rows. Each pel's coordinate is the current frame is estimated by searching through a larger 

defined by an intersection of a vertical column with a search window in the previous frame for a best match. At 

horizontal row. In converting each frame of video into a each possible shift position within the search window, a 

frame of digital video data, scan lines of the two interlaced 60 pel-by-pel absolute comparison is made between the inten- 

fields making up a frame of un-digitized video are interdigi- sity of the pels in the block in the current frame and the 

tated in a single matrix of digital data. Interdigitization of the corresponding pels in the previous frame. Each pel is 

digital video data causes pels of a scan line from an odd-fleld classified as either a matching pel or a mismatching pel 

to have odd row coordinates in the frame of digital video depending on the pel difference and a threshold. The number 

data. Similarly, interdigitization of the digital video data 65 of matching pels at each possible shift position is counted 

causes pels of a scan line from an even-field to have even and the motion vector is determined from the shift position 

row coordinates in the frame of digital video data. that yields the mmHmiim number of matching pels. 
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FIGS 3a through 3c show representative steps in a absolute difference (MAD) between a macroblock 16 and a 

full-search block-matching motion estimation method. In corresponding block in a search region. These calculations 

PIG. 3a. a 16x16 pel block 102 in the current frame (not can be reduced by using ; ■ ^ hierarchical inetood of search 

illustrated in any of fee FIGS.) is shown being compared app^d to filtered decimated macroblocks and filtered deci- 

with the upper-most and left-most corresponding position in 5 mated search regions After V^mr *» P^J-V 

*[ , a1c _ nrrf search, a search is performed on undeomated pels making 

a search window 104 of the reference faune (alsonot • decimal match. This method of hierarchical 

illustrated to any of the FIOS.) and moving to the upper- P ^ ^ ^ a m ^ eudk 0Vff , smaSla 

most and nght^ost position in the tench window 104 of efficiently and accurately determine motion vec- 

the reference frame. In FIG. 36. the block 102 in the current ^ 

frame is shown moved down one row in the search window 10 ' . , ... 

ujiBD»™«»iTuii»;mi™i, wn A new method of motion estimation in accordance with 

104 of thereference frame from the imnal r^sidon shx,wn in ^ of ^ Nation includes the hierar- 

FIG. 3a. The progression conhnuesdowr^e search window ^^^^ scarch . ^ hierarchical search 

104 row by row un^. as shown m HG 30. 102 in computations required for motion estimation, 

mecurrentframeultimatelyis^ faste/than does the full-search block-matching 

rightmost corresponding position in me search wmdow 104 is ^ ^ ^ ^ ^ a m ^ 

from me reference frame. . quaUr y. The hierarchical search uses block-matching tech- 

The block-matching method disclosed in the Gharavi <*> ^ rfOTming a block search on a large area at 

patent is effective for reducing the computational oad of g hoarse resolutioa. and then by performing a second block 

motion estimation, without sacrificing video quality, as search on a smaller area within the large area, and at a finer 

compared to a thorough pel-by-pel search across the entire »> resolution 

frame. However, as can be deduced from FIGS. 3o through fuu.^ch block-matching and hierarchical block- 

3c. the Mc<k-matcrdng procedure still relies on a^rute- J^^JSods represent two* approaches to MPEG-2 

force" honzontal. pd-by-peL and vertical. Une-by-Une. 8^ ^ somewhat opposite ends of a 

search/comparison of the block in the current frame with vwsus quaUty spectrum. X: full-search 

corresponding block areas in the search window of the * P * £ £ precise motion 

reference frame. The disclosed block-matching procedure ^£ 0 ° n but at a performance cost The hierarchical 

therefore requires that the block in me current^ be hand, provides faster searching while 

compared, pel-by-pel with every possible block position occasioiially finding a less than optimum motion 

within the reference frame's search ^Tbe present invention provides a mLs to incorpo- 

senting an Lmproyement in comr^tation^r^orm^ce oyer *> ^ MPEG-2 encoding system, to 

a pure pel-by-pel comparison, the rwrnod d^c^ed ui toe ^ tQ ^ rfof . 

Gr^vi^teDt leaves r«>mfor^^ ^ «*J ^ reduce me 

computation r^uirements. without noticeably sacrificing ^ h i owa re required, 

video quality. The disclosure of the Gharavi patent is Incor- _ . ^^"^^ * . . , . . . 

^tJTrJu k« ™.f«™~ w The decision of whether to perform a hierarchical search 

porated herein by reference. or & ^ aMy on ^ c type of frame 

SUMMARY OF THE INVENTION currently being processed, i.e. P-frame or B-frame. and, in 

, . f lU . . . ^ „ the case of a B -frame, the distance of the B-frame being 

An object of the present invention is to provide an , , . . , * r ^ ™" 

Jt J * . \. j r • ijmcr o processed from the frame being used as a reference. The 

efficient motion estimation procedure for use in MPEG-2 _ . . A „ Mt f ui „l „ u;^. 
7* _ t 40 deasion to employ a full-search block-matching or a nier- 

vidco encoding systems. archical bic*k-matdiing scarch is based on the resources. It. 

Another object of the present invention is to provide an Ume and computing hardware, required to perform the 

efficient motion estimation procedure that can take advan- search For scarcn framcs » closcr » to the reference frames), 

tage of the higfr resolution capabihty of a full-search block- fl 0^^^ full-search can be performed with the 
niatching motion estimation method. 45 hardware available and in the required time. For search 

Yet another object of the present invention is to provide an frames from me reference frame(s), a hierarchical 

efficient motion estimation procedure that can take advan- scarch wiil ^ performed to conserve computing hardware 

tage of the high performance capability of a hierarchical afld mc sea rch within the required time, 

block-matching motion estimation method, ^ additlon to accurately and efficiently estimating motion 

Yet another object of the present invention is to provide an ^ oycr Qjeas. the preferred embodiment of the present 

efficient motion estimation procedure that can take advan- invention similarly accurately and efficiently conforms 

tage of both the high resolution capability of a full-search motion estimation both to interlaced and to non-interlaced 

block-matching motion estimation method and the high video. Field motion estimation is best suited to interlaced 

performance capability of a hierarchical block-matching video. Conversely, frame motion estimation is best suited to 
motion estimation method. 55 non-interlaced video. However, moment by moment the 

Yet another object of the present invention is to provide an type of video, i.e. interlaced or non-interlaced video, 

efficient motion estimation procedure both for interlaced and received by an encoder chip may change back and forth. To 

for non-interlaced video. accommodate both types of video, the preferred embodi- 

Yet another object of the present invention is to provide an ment of the present invention concurrently determines five 
efficient motion estimation procedure that can provide an 60 (5) different independent motion vectors, i.e. a frame motion 

optimum blend of video quality and processor performance vector, an odd field— odd field motion vector, an even 

by taking advantage of both the high resolution capability of field — even field motion vector, an odd field— even field 

a full-search block-matching motion estimation method and motion vector, and an even field— odd field motion vector, 

the high performance capability of a hierarchical block- A significant difference between a macroblock's field 
matching motion estimation method. 65 motion vectors and its frame motion vector indicates that the 

MPEG video requires billions of subtraction and addition digital video data being processed by the encoder chip is 

operations per second for calculations of the minimum interlaced rather than non-interlaced. 
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These and other features, objects and advantages will be video data 82 by applying medial, spatial and temporal 
understood or apparent to those of ordinary skill in the art filters (not illustrated in any of the FIGS.) to that data. The 
from the following detailed description of the preferred Video Interface 54 also accepts, detects and eliminates 3/2 
embodiment of the invention as illustrated in the various pulldown. Finally, the Video Interface 54 may also indepen- 

drawing figures. 5 dently generate genlock. 

..... ^ T ^ ^ n ATTrTV7 ^ n Referring to both FIGS. 1 and 2, MFEG-2 encoded video 

BRIEF DESCRIPTION OF THE DRAWINGS ^ of G0Fs 1#t ^ 10 of one or 

FIG. 1 is a block diagram showing the coding format of more I-frames together with all of the P- frames and 

a video image according to an MFEG-2 protocol. B-frames for which the I-frames serve as a reference. An 

nG.2isablockdiagramshowmganencodingsystemfor 10 MFEG-2 I-frame includes sufficient data to reproduce an 

generating video signals using the MPEG-2 protocol shown entire uncompressed frame of video data without reference 

^ pjQ | to any other data. An MPEG-2 P-frame may be decoded to 

r^^n ' A . . - . - „ . obtain an entire uncompressed frame of video data only by 

* ? ? representauve s^ in a priorTcoded I-frame or by reference to a 

full-search block-matching moUon estimation procedure. 15 ^ ^^ P p _frame. An MPEG-2 B-frame may be 

FIGS. 4a through 4c show a pictorial overview of a decoded to obtain an entire uncompressed frame of video 

hierarchical block-matching motion estimation procedure. ^ by reference t0 on i y a prior reference frame, to only a 

FIG. 5 shows a flow chart depicting a process for deter- subsequent reference frame, or both to a prior and to a 

niining whether a hierarchical or a full-search will be used subsequent reference frame, i.e. By reference to a decoded 

in determining motion vectors for a frame's macroblocks. 20 I-frame or P-frame(s). 

FIG. 6 illustrates a portion of a search area including odd One of the processing steps required for encoding video 

numbered and even numbered rows of pels that are overlaid data into P-frames and B-frames includes subdividing the 

with an outline of a macroblock which encompasses a digitized video data into macroblocks b6. Each macroblock 

(16x16) pel block within the search area. 16 consists of a 16x16 pel array of digitized video data. The 

FIGS. 7a and 7b graphically depict the temporal relation- 25 Motion Estimator 56 included in the MPEG-2 Encoder Chip 

ships existing among I or P-frames and a B-frame that are to 52 determines the direction and amount of motion between 

be processed for motion estimation. macroblocks 16 belonging to different frames of the video 

data being encoded. 

DETAILED DESCRIPTION OF THE ^ Tbe Encoding Decision Block 66 performs macroblock 

PREFERRED EMBODIMENT 30 w mtra/intei> ,Q decisions, makes field and frame decisions, 

Referring to FIG. 2, an MPEG-2 encoding system, and performs rate control, half-pel motion estimation and 

referred to by the general reference character 50. is shown video buffer verifier (VBV) calculations, 

which employs the hybrid hierarchical/full-search block- The Encode Pipe 58 encodes macroblocks 16. Based 

matching motion estimation procedure of the present inven- 33 either on results from the Encoding Decision Block 66, or 

tion. based upon parameters specified via the Host Interface 94, 

The Encoder Chip 52 accepts raw* digitized video data 82 the Encode Pipe 58 accepts digitized video data 82 together 

in most standard formats and produces an MPEG-2 com- with data produced by the Encoding Decision Block 

pressed video digital data stream (not illustrated in any of the While the Motion Estimator 56 determines motion vectors 

FIGS.). The Encoder Chip 52 also accepts digital audio data 40 and the Encoding Decision Block 66 selects an encoding 

86 compressed in accordance with the MFEG-2 standard mode by processing 16x16 macroblocks 16, the Encode Pipe 

and combines the MPEG-2 compressed digital audio data 86 58 transforms the digitized video data and quantizes the 

and compressed video data to produce an MPEG-2 System transformed video data by processing 8x8 blocks 18 and 20. 

Stream 84. Each 8x8 block 18 or 20 is transformed by computing DCT 

The system components performing the functions neces- 45 coefficients, and then the coefficients thus computed are 

sary to produce the MPEG-2 System Stream 84 include: a quantized to further reduce the amount of data. If an inter 

Video Interface 54, Motion Estimator 56, Encode Pipe 58, macroblock cannot be adequately encoded simply by motion 

Video System Stream Multiplexor 62, Output Stream Inter- vectors alone, a DCT is computed of the difference between 

face 64, Encoding Decision Block 66, Audio Interface 68, its reference frame(s) or field(s) and the macroblock's video 

Main DRAM Controller 92 and Host Interface 94, all 50 data, and the DCT coefficients obtained for this difference 

contained on the Encoder Chip 52; a Host Computer 78; a macroblock are quantized. If P-frame(s) provide the differ- 

Host DRAM 76; and a Main DRAM 88 which includes a ence for an inter macroblock. then the DCT is computed of 

Bank A DRAM 72 and a Bank B DRAM 74. the difference between the macroblock' s video data and the 

The Video Interface 54 accepts digitized video data 82 decoded P-frame data to obtain the maximum accuracy in 
directly into the MPEG-2 Encoder Chip 52 and may filter the 55 the macroblock's encoding. The transformed, quantized data 
digitized video data to reduce noise and to smooth the image thus obtained are then coded efficiently into an MPEG-2 
by an amount appropriate for the resolution being used In video stream using variable length coding (VLC). Huffman 
particular, the noise reduction and smoothing provided by coding. The Encode Pipe 58 also performs the inverse of 
the Video Interface 54 permits better matching between a these operations simultaneously on P-frame encoded data, 
macroblock 16 and a block in a search area. The MFEG-2 60 An MPEG-2 system stream 84 consists primarily of 
Encoder Chip 52 accepts component coded video data interleaved packets of data extracted from an MPEG-2 
consisting of luminance (Y), and two color difference Cb compressed video bitstream and from an MPEG-2 com- 
and Cr. The Video Interface 54 may decimate the component pressed digital audio data 86 bitstream together with header 
video data thereby reducing the amount of data by combin- data that is essential for correctly decoding the compressed 
ing data values for adjacent pels, i.e., the Video Interface 54 65 bitstreams. The MFEG-2 system stream 84 may also include 
may perform a 4:2:2 to 4:2:0 decimation. The Video Inter- interleaved packets of auxiliary data together with the pack- 
face 54 may be programmed to reduce noise in the digitized ets extracted from the MPEG-2 compressed digitized video 
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data 82 and digital audio data 86 bitstreams. The Video 
System Stream Multiplexer 62 packs header data around 
packets of data extracted from the MPEG-2 compressed 
digitized video data 82 and digital audio data 86. The Video 
System Stream Multiplexor 62 gathers header data for each 
packet of MPEG-2 compressed video from other compo- 
nents of the encoder chip 52. and generates data required for 
the MPEG-2 system stream's 84 system header and pack 
header. 

The Audio Stream Interface 68 receives and buffers 
digital audio data 86 compressed in accordance with the 
MPEG-2 standard, before such compressed digital audio 
data 86 is combined with the MPEG-2 compressed video 
data in the Video System Stream Multiplexor 62. 

In applications for the MPEG-2 Encoder Chip 52 which 
require transmission of a serial bitstream, the Output Stream 
Interface 64 generates and supplies such a serial bit stream. 
The Output Stream Interface 64 may supply a serial bit- 
stream consisting of an MPEG-2 compressed digital audio 
data 86. an MPEG-2 compressed video stream, or a serial 
MPEG-2 system stream 84. The Output Stream Interface 64 
supplies the selected bitstream at a rate specified by a clock 
which may either be internal or external to the MPEG-2 
Encoder Chip 52. In generating and supplying the serial 
bitstream. the Output Stream Interface 64 automatically 
retrieves buffered MPEG-2 system stream, MPEG-2 Video 
stream, or MPEG-2 audio stream data from the host DRAM 
76. 

The Host Interface 94 controls timing and access of the 
MPEG-2 Encoder Chip 52 to the Host DRAM 76, and also 
provides a path by which the Host Computer 78 may 
supervise and control the Encoder Chip's 52 operation. The 
Host Interface's 94 ability to access the Host Dram 76 
provides the Encoder Chip 52 with a path to the Host Dram 
76 for storing MPEG-2 system stream 84 output data, and 
for receiving MPEG-2 auxiliary stream input data and/or 
MPEG-2 audio stream data 

The Main Dram Controller 92 provides timing both for 
overall control of the Encoder Chip 52. and controls the ^ 
operation of a dedicated Main DRAM 88 that is accessed by 
the Encoder Chip 52 via a 64 bit wide bus. The Main DRAM 
Controller 92 includes a smart sequencer that automatically 
accesses the dedicated Main DRAM 88 to supply data as 
needed to the other components included on me MPEG-2 
Encoder Chip 52. To provide the most efficient storage of 
pictures together with sufficient bandwidth for motion 
estimation, the dedicated Main DRAM 88 has two indepen- 
dent banks of DRAM, Bank A 72 and BankB 74. Each Bank 
address accesses 8 pels of stored digitized video data, Le. 
each address in the combined Bank A 72 and Bank B 74 
accesses one macroblock 16 row of digitized video data. 
Both Bank A 72 and Bank B 74 of the Main DRAM 88 
respectively include four (4) sets of DRAMs interleaved so 
sequential Bank addresses successively access a different set 
of DRAM. The MPEG-2 Encoder Chip 52 uses the Host 
DRAM 76 as a ninth DRAM primarily for intermediate 
storage (buffering) of encoded streams between their encod- 
ing and their output from the MPEG-2 Encoder Chip 52. 

In the present invention, operating parameters are passed 
to the MPEG-2 Encoder Chip 52 from a Host Computer 78. 
The operating parameters specify how the Encoder Chip 52 
should search for the motion vectors. If the operating 
parameters specify a hierarchical search, for each macrob- 
lock 16 the MPEG-2 Encoder chip scans up to a "64 pel 
search area in the reference video data at a coarse resolution, 
and then scans a smaller area at full resolution. If the 



operating parameters specify a full-search motion vector 
search, the MPEG-2 Encoder Chip 52 scans up to a ±32 pel 
search area in the reference video data. If the search range 
specified by the operating parameters falls into the ±32 pel 
search area range of the motion estimation hardware, the 
full-search occurs. Otherwise, the MPEG-2 Encoder Chip 52 
performs the 2 step hierarchical search. The operating 
parameters may specify different search ranges for any 
combination of field, frame, different B-frames which indi- 
cates a distance from P-frames or I-frames. since every 
possible operating condition may not require processing the 
entire search area. 

FIGS. 4a. 4b and 40 provide a pictorial overview of the 
hierarchical search method. In FIG. 4a, an nxn search area 
112, e.g. 128x128 in a reference frame, is filtered and 
decimated down to an (n/2)x(n/2) hierarchical search area 
114. An mxm macroblock 16, e.g. 16x16 is likewise filtered 
and decimated down to an (m/2)x(ro/2) hierarchical search 
block 118. FIG. Ab illustrates performing a block search 
20 within the filtered and decimated (n/2)x(n/2) hierarchical 
search area 114 using the (m/2)x(m/2) hierarchical search 
block 118. The search of the (n/2)x(n/2) hierarchical search 
area 114 identifies the area that provides the best match for 
the (m/2)x(m/2) hierarchical search block 118. Because the 
original search area 112 has been filtered and decimated 
from an nxn area to the (n/2)x(n/2) hierarchical search area 
114, and the mxm macroblock 16 has likewise been reduced 
to an (m/2)x(m/2) hierarchical search block 118. this first 
search yields a result having only one-half the resolution of 
the original nxn search area 112 and mxm macroblock 16. 

Referring to FIG. 4c, an mxm region 122 within the nxn 
search area 112 corresponding to the (m/2)x(m/2) region in 
the (n/2)x(n/2) hierarchical search area 114 that provides the 
best match for the hierarchical search block 118 is then 
searched for the best match to the mxm macroblock 16. The 
region in the search window 104 that is searched is an 
(m+2)x(m+2) augmented block 126 surrounding the mxm 
region 122. The area to be searched is expanded from m*m 
to <m+2)x(m+2) because each pel in the filtered and deci- 
mated (n/2)x(n/2) hierarchical search area 114 represents a 
2x2 pel area in the original nxn search area 112. Thus, when 
expanded, the pels in the rows and columns immediately 
adjacent to the mxm block 122 must be included in the 
second, full resolution search. 

The resultant motion vector for the mxm macroblock 16 
is equal to the difference in the position of the mxm 
macroblock 16 in the current frame and the mxm block in 
the reference frame which best matches the mxm macrob- 
lock 16 in the current frame. 

Table 7-8 of the ISO MPEG-2 decoder specification, 
reproduced below, lists a set of maximum sizes for motion 
vectors that may be coded in an MPEG-2 video stream. 
Table 7-8 defines a set of f_codes. i.e. 1 through 9, which 
respectively specify the size of an area in the reference frame 
that is to be searched in determining a motion vector. 
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go L_code[f][t] 



\tertical components (t=l) of 
field vectors in frame pictures 



All other cases 



65 



(faibkttcn) 
H*: +3.5] 
H8: +7.5] 
[-16: +15.5] 
1-32: +31.3] 
[-64: +63.5) 
1-128: +127,5] 



[-8: +7 JS) 
[-16: +15.5] 
[-32: +31.51 
[-64: +63.5] 
[-128: +127.5] 
[-256: +255.5] 
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<-«xfc[ S ]M 


Vertical components (t=l) of 
field vectors in frame pictures 


All other cases 


7 


[-256: +255 3} 


[-512: +511.5] 


8 


[-512: +511.5] 


(-1024: +1023.5] 


9 


[-1024; +1023.5] 


[-2048: +2047.5] 


10-14 


(reserved) 




13 


(Used when a particular (_code[s][tJ will not be used) 



Id the presently preferred embodiment of the present 
invention the maximum search range for motion estimation 
is "64 pels. i.e. f_code=4. Searches covering ranges greater 
than ±32 pels use the hierarchical block-matching method, 
and searches covering ranges less than or equal to ±32 pels 
use a full-search block-matching method such as that dis- 
closed in the Gharavi patent. These preceding limits result 
from the performance of the Encoder Chip 52 and amount of 
Main DRAM 88 available to support motion estimation. 



15 



FIG. 7a graphically depicts the temporal relationship 
existing between a P-frame 252 to be processed in the 
forward direction far motion estimation with respect to a 
reference I-frame or P-frame 254. FIG. lb graphically 
depicts the temporal relationships existing between a 
B-frame 256 to be processed for motion estimation in the 
forward direction with respect to the reference I-frame or 
P-frame 254* and also in the backward direction with respect 
to the second reference P-frame 252. 

Thus as depicted in the flow chart of PIG. 5, the Motion 
Estimator 56 commences processing a current frame 252 or 
256 of digital video data with respect to a particular refer- 
ence frame 254 or frames 252 and 254 at a start block 202. 
In processing an entire frame of digital video data in the 
forward direction with respect to the reference frame 254, 
the Motion Estimator 56 first identifies the frame currently 
being processed either as the P-frame 252. or as the B-frame 
256 in a decision block 204. If the current frame 252 or 256 
is the B-frame 256, the Motion Estimator 56 then determines 



Greater search ranges may be used in accordance with the . ^ . m - fnmtarti 

principles of the present invention as the performance of *> m * ***** of ^ ^°J*A C for »* d 

encoder chips improves and the size and cort of memory reference *™ 254 in a processing block 206. Regardless 
decreases. 

The Motion Estimator 56 includes a group of five (5) 
f_code specification registers (not illustrated in any of the 



of whether the current frame is the P-frame 252 ox the 
B-frame 256. in a processing block 208 the Motion Estima- 
tor 56 fetches a forward direction f_code from the appro- 



nGS.)wm^storeasetoffcodesr^^ 25 priate f.code specification register as described above. 



specifications are downloaded to the Encoder Chip 52 from 
the Host Computer 78 during system initialization. The 
following table lists the correlation between each f code 
specification register and the control over motion estimation 
exercised by an f_code stored into such f_ code specifica- 
tion register. 



Register 


Applicable 


Distance from 


Name 


Frame 


Reference Frame 


f_code_0 


P-frame 


(n/a) 


C_code_l 


B-frame 


+1 tmd-1 


f_code_2 


B-frame 


+2 and -2 


t_code_3 


B-frame 


+3 and -3 


C_code 4 


B- frame 


+4 and -4 



Then in a processing block 212 the Motion Estimator 56 
uses the retrieved forward direction f_code to establish a 
forward direction search range for macroblocks 16 of the 
current frame 252 or 256 in accordance with the ranges 
30 specified in Table 7-8 of the ISO MPEG-2 specification set 
forth above. If in a decision block 214 of the flow chart of 
FIG. 5 the Motion Estimator 56 determines that the search 
range thus established for the current frame 252 or 256 
exceeds a pre-established threshold, then forward motion 
vectors for all the macroblocks 16 of the current frame 252 
or 256 are determined using a hierarchical search in pro- 
cessing block 218. If. however, the search range established 
fox the current frame 252 or 256 is less than or equal to the 
pre-established threshold, then the Motion Estimator 56 
fctermines forward motion vectors for all the macroblocks 
16 of the present frame using full-search as depicted in a 
processing block 216. 

After all the macroblocks 16 in the current frame 252 or 
256 have been processed either in block 216 or in block 218, 



35 



40 



As indicated in the preceding table, the f_code_0 reg- 
ister stores the f_code that is used to determine the motion 
estimation search range for all P-frames. The f_code_l 

register stores the f_codes used to determine the motion 45 the Morion Estimator 56 then determines in decision block 



estimation search range for B-frames which are located 
either 1 frame before or 1 frame after the reference frame. 
Likewise, the f_code__2, f_code_3 and f_cod_4 registers 
store the f_codes used to determine the motion estimation 
search ranges for B-frames which are located 2, 3 or 4 
frames, respectively, from (preceding or following) the 
reference frame. In the embodiment of the present invention, 
B-frames will always be within 4 frames of a reference 
frame. Also, in the embodiment of the present invention, two 



224 whether the present frame is the B-frame 256 that has 
been processed only in the forward direction. If the present 
frame is the B-frame 256 that has been processed only in the 
forward direction, then the Motion Estimator 56 in process- 
50 ing block 226 determines the distance of the current frame 
256 from the backward reference frame 252. Then in pro- 
cessing block 228 the Motion Estimator 56 fetches a back- 
ward direction f_code from the appropriate f_code speci- 
fication register as described above. In a processing block 



f__code values are stored in each of the f_code specification 55 232 the Motion Estimator 56 uses the retrieved backward 



registers f_codc_l, f__codc_2, f_code_3, and f_codc_4. 
Three bits of each f_code specification register identify the 
f— code to be used if the B-frame precedes the reference 
frame, and three bits identify the f__code to be used if the 
B-frame follows the reference frame. Because during ini- 
tialization of the Encoder Chip 52 any valid fl_code value 
may be loaded into the f_codc specification register both for 
forward and for backward searching, a different f_code 
value may be specified for a forward search that is **n" 
frames ahead of the reference frame than the f-code value 
specified for a backward search that is "n" frames behind the 
reference frame. 



60 



65 



direction f_code to establish a backward direction search 
range for macroblocks 16 of the current frame 256 in 
accordance with the ranges specified in Table 7-8 of the ISO 
MPEG-2 specification set forth above. If in the decision 
block 214 of the flow chart of FIG. 5 the Motion Estimator 
56 determines that the search range thus established for the 
current frame 256 exceeds the pre-established threshold, 
then backward motion vectors for all the macroblocks 16 of 
the current frame 256 arc determined using a hierarchical 
search in processing block 218. If, however, the search range 
established for the current frame 256 is less than or equal to 
the pre-established threshold, then the Motion Estimator 56 
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determines backward motion vectors for all the macroblocks 
16 of the present frame using full-search as depicted in a 
processing block 216. 

After the P-frarae 252 has been processed only in the 
forward direction, or the B-framc 256 has been processed 
both in the forward and in the backward directions, the 
Motion Estimator 56 exits the motion estimation process for 
the current frame 252 or 256 via an exit block 222. In the 
presently preferred embodiment of the invention, Ihe pre- 
established threshold is set at a search range of ±32 pels. 

Those skilled in the art are familiar with various tech- 
niques for comparing pels during motion estimation search- 
ing in order to identify the block in the reference frame that 
most closely matches the block in the current frame. This 
includes techniques such as: simply comparing pel-by-pel 
and counting the number of absolute matches or mis- 
matches; comparing the intensity of pels to determine the 
difference In their intensities, and men comparing the dif- 
ference in intensity of the pels with a threshold to determine 
if the pels match or mismatch; taking a sum of absolute 
differences between pels to find a minimum sum of absolute 
difference between a macroblock 16 and a block within a 
search window; or taking a sum of squared differences 
between pels to find a minimum sum of squared difference 
between a macroblock 16 and a block within a search 
window. The use of the hybrid hierarchical/full-search 
block-matching motion estimation method of the present 
invention is compatible with these and other pel comparison 
methods used in motion estimation. 

However, the preferred embodiment of the Motion Esti- 
mator 56 of the present invention computes sums of absolute 
differences ('*SADs M ) both between pels of the hierarchical 
search block 18 and pels of the hierarchical search area 114, 
and between pels of the macroblock 16 and pels of the 
search area 112. Referring now to FIG. 6, depicted there are 
pels of a portion of the search area 112 overlaid with an 
outline of the macroblock 16 which encompasses a (16x16) 
pel block within the search area 112. In the illustration of 
FIG. 6, pels in rows 304 belonging to an odd field are 
depicted by open circles, while pels in rows 306 belonging 
to an even field are depicted by filled circles. In computing 
various SADs far the macroblock 16 not only does the 
Motion Estimator 56 determine a motion vector ( ,< MST) for 
the minimum SAD for the entire macroblock 16, in search- 
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4. the even-odd field, i.e. the even rows 306 of the search 
area 112 and the odd numbered rows of the macroblock 
16. 

In computing SADs during full-search at each successive 
position of the macroblock 16 within the search area 112 or 
within the augmented block 126, the Motion Estimator 56 
independently cumulates the SAD for the odd-odd field and 
the SAD for the even-even field and then merely adds these 
two SADs together rather than independently computing a 
SAD for the entire macroblock 16 at each such position. For 
the odd-even field and the even-odd field, the Motion 
Estimator 56 shifts the macroblock 16 down one row, as 
illustrated by a dashed line outline position 16' for the 
macroblock, and independently determines SADs for the 
odd-even field and the even-odd field. Again the Motion 
Estimator 56 adds the SAD for the odd-even field to the 
SAD for the even-odd field to obtain a SAD for the entire 
macroblock at the position 16'. While determining SADs at 
successive positions of the macroblock 16 throughout the 
search area 112 or throughout the augmented block 126, the 
Motion Estimator 56 maintains two independent minimum 
SADs and motion vectors for the macroblock 16 both at 
those positions where the SAD is computed by adding 
together odd-odd field and even-even field SADs, and at 
those positions where the SAD is computed by adding 
together odd-even and even-odd SADs. Only after exhaust- 
ing the search for tninimum SADs throughout the search 
area 112 or throughout the augmented block 126 does the 
Motion Estimator 56 select the minimum of these two SADs 
and motion vectors as being the minimum SAD and motion 
vector for the entire macroblock 16. 

Consequently, after processing each macroblock 16 of the 
P-frame 252 throughout the search area 112 of the reference 
frame 254. the Motion Estimator 56 stores into the Host 
DRAM 76 for subsequent use by the Encoding Decision 
Block 66 only the forward MVs and SADs listed in the table 
below. Correspondingly, after processing each macroblock 
16 of the B-frame 256 throughout the search areas 112 of the 
reference frames 252 and 256 the Motion Estimator 56 
stores into the Host DRAM 76 for subsequent use by the 
Encoding Decision Block 66 both the forward and the 
backward MVs and SADs listed in the table below. 



Forward Backward 
P-frame or B-fora> B-frame 

Mm. SAD Hor. Vert Mm. SAD Hot: Vert 



Entire Macroblock SAD m MVHm MW m SAD. MVH*, MW M 

Odd-Odd Field SAD^ MVHq-o MWo-o SATVo MVHo-o MW^ 

Evcd-Bycd Field SATVq MVHo^ MW^b SADb_ b MVHb_ b MW w 

Odd-Even Field SADo- B MVHo_ 8 MW^, SAD^ MVHo-b MWo_b 

Eveo-Odd Field SATVo MVHm MW m SADb_o MVHb_o MW m 



ing for the best match for pels of the macroblock 16 the A significant difference between the motion vectors for 

Motion Estimator 56 also concurrently determines indepen- macroblocks 16 and those for the odd-odd field and the 

dent minimum SADs and MVs for: ffl even-even field of the macroblocks 16 indicates that this is 

1. the odd-odd field, i.e. the odd rows 304 of the search probably a frame digitized from an interlaced video signal, 
area 112 and odd numbered rows of the macroblock 16; Conversely, little difference between the motion vectors for 

2. the even-even field, Le. the even rows 306 of the search macroblocks 16 and those for the odd-odd field and the 
area 112 and the even numbered rows of the macrob- even-even field of the macroblocks 16 indicates that this is 
lock 16; probably a frame digitized from a non-interlaced video 

3. the odd-even field. Le. the odd rows 304 of the search 65 signal. 

area 112 and the even numbered rows of the macrob- Note mat if hierarchical search has been used in process- 
lock 16; and ing block 218 of the flow chart presented in FIG. 5 to 
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determine motion vectors for all the macroblocks 16 of the 
frame 252 or 256, (he motion vectors tabulated above for 
each macroblock 16 must all lie within the augmented block 
126 depicted in FIG. 4c. That is, in the presently preferred 
embodiment of the present invention the maximum differ- 
ence respectively among the horizontal and among vertical 
components of all motion vectors determined for raacrob- 
locks 16 using hierarchical search cannot exceed two (2). 
Alternatively, in the presently preferred embodiment of the 
present invention the maximum difference respectively 
among the horizontal and among the vertical components of 
all motion vectors determined for macroblocks 16 using full 
search in the flow chart processing block 216 of FIG. 5 can 
be as large as sixty-three (63). 

Although the present invention has been described in 
terms of the presently preferred embodiments, it Is to be 
understood that such disclosure is purely illustrative and is 
not to be interpreted as limiting. Consequently, without 
departing from the spirit and scope of the invention, various 
alterations, modifications, and/or alternative applications of 20 
the invention will, no doubt be suggested to those skilled in 
the art after having read the preceding disclosure. 
Accordingly, it is intended that the following claims be 
interpreted as encompassing all alterations, modifications, or 
alternative applications as fall within the true spirit and 25 
scope of the invention. 

What is claimed is: 

1. In a digital video data compression scheme, a method 
of estimating the frame -to-frame motion of a block of pels 
in a video frame comprising the steps of: 30 

(a) filtering and decimating a block located at a position 
in a current frame thereby establishing a hierarchical 
search block; 

(b) forming a search area in a reference frame, with said 
search area in said reference frame surrounding the 33 
position of said block in said current frame; 

(c) filtering and decimating said search area in said 
reference frame thereby establishing a hierarchical 
search area; ^ 

(d) comparing said hierarchical search block with a block 
within said hierarchical search area; 

(e) repeating step (d) for a plurality of blocks located at 
various positions within said hierarchical search area; 

(f) identifying that region within said hierarchical search 45 
area which best matches said hierarchical search; 

(g) forming an augmented block within said search area in 
said reference frame that surrounds a region in said 
search area in said reference frame that corresponds to 
said region within said hierarchical search area mat was 50 
identified in step (f) as best matching said hierarchical 
search block; 

(h) comparing the pels in said block in said current frame 
with pels in a block in said augmented block in said 55 
reference frame; 

(i) repeating step (h) for a plurality of blocks located at 
various positions within said augmented block in said 
reference frame; and 

(j) establishing a motion vector for said block in said 60 
current frame equal to a difference between a position 
of that block in said current frame and a position in said 
augmented block in said reference frame which best 
matches said block in said current frame. 

2. The method of estimating frame-to-frame motion of a 65 
block of pels in a video frame of claim 1 wherein said 
plurality of blocks located at various positions within said 



,850 
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hierarchical search area includes all blocks within said 
hierarchical search area. 

3. The method of estimating frarnc-to-framc motion of a 
block of pels in a video frame of claim 1 wherein said 

5 plurality of blocks located at various positions within said 
augmented block includes all blocks within said augmented 
block. 

4. The method of estimating frame- to-frame motion of a 
block of pels in a video frame of claim 1 wherein a plurality 
of motion vectors are concurrently established including an 

10 odd-odd field motion vector, an even-even field motion 
vector, an odd-even field motion vector, and an odd-even 
field motion vector. 

5. In a digital video data compression scheme, a method 
of estimating the frame-to- frame motion of a block of pels 
»« a video frame comprising the steps of: 

(a) identifying a reference frame; 

(b) identifying a block located at a position in a current 
frame; 

(c) determining a size for a search area in said reference 
frame; 

(d) if the size of said search area in said reference frame 
is larger than an established threshold, then 
(i) filtering and decimating said block in said current 

frame thereby establishing a hierarchical search 
block; 

(u) forming a search area in said reference firame, with 
said search area in said reference frame surrounding 
the position of said block in said current frame; 
(hi) filtering and decimating said search area in said 
reference frame thereby establishing a hierarchical 
search area; 

(iv) comparing said hierarchical search block with a 
block within said hierarchical search area; 

(v) repeating step (d) (iv) for a plurality of blocks 
located at various positions within said hierarchical 
search area; 

(vi) identifying that region within said hierarchical 
search area which best matches said hierarchical 
search block; 

(vii) forming an augmented block within said search 
area in said reference frame that surrounds a region 
in said search area in said reference frame that 
corresponds to said region within said hierarchical 
search area that was identified in step (d) (vi) as best 
inarching said hierarchical search block; 

(viii) comparing the pels in said block in said current 
frame with pels in a block in said augmented block 
in said reference frame; 

(ix) repeating step (d) (viii) for a plurality of blocks 
located at various positions within said augmented 
block in said reference frame; and 

(x) establishing a motion vector for said block in said 
current frame equal to a difference between a posi- 
tion of that block in said current frame and a position 
in said augmented block in said reference frame 
which best matches said block in said current frame; 

(e) otherwise, if the size of said search area in said 
reference frame is equal to or less than said established 
threshold, then 

(i) forming a search window in said reference frame, 
with said search window in said reference frame 
surrounding the position of said block in said current 
frame; 

(il) comparing the pels in said block in said current 
frame with pels in a block within said search window 
in said reference frame; 
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(iii) repeating step (e) (ii) for a plurality of blocks 
located at various positions within said search win- 
dow in said reference frame; 

(iv) establishing a motion vector for said block in said 
current frame equal to a difference between a posi- 
tion of that block in said current frame and a position 
in said search window in said reference frame which 
best matches said block in said current frame. 

6. The method of estimating frame-to- frame motion of a 
block of pels in a video frame of claim 5 wherein said 
plurality of blocks located at various positions within said 
hierarchical search area includes all blocks within said 
hierarchical search area. 

7. The method of estimating frame-to-frame motion of a 
block of pels in a video frame of claim 5 wherein said 
plurality of blocks located at various positions within said 
augmented block includes all blocks within said augmented 
block. 

8. The method of estimating frame-to-frame motion of a 
block of pels in a video frame of claim 5 wherein said 
predetermined number of blocks within said search window 
in said reference frame comprises all blocks within said 
search window in said reference frame. 

9. The method of estimating frame-to-frame motion of a 
block of pels in a video frame of claim 5 wherein a plurality 25 
of motion vectors are concurrently established including an 
odd-odd field motion vector, an even-even field motion 
vector, an odd-even field motion vector, and an odd-even 
field motion vector. 

10. The method of estimating frame-to-frame motion of a 30 
block of pels in a video frame of claim 5 wherein said 
method of determining the size for a search area in said 
reference frame comprises the steps of: 

(a) assigning a code to said current frame; and 

(b) detenwriing the size of said search window based on 
the search range specified for said code. 

11. The method of estimating frame-to-frame motion of a 
block of pels in a video frame of claim 10 wherein said step 
of assigning a code to said current frame comprises the steps 
of: 

(a) obtaining a code from a table of codes; and 

(b) associating said code with said current frame. 

12. The method of estimating frame-to-frame motion of a 
block of pels in a video frame of claim U wherein said step 45 
of obtaining a code from a table of codes comprises the steps 
of: 

(a) determining whether said current frame is a P-frame or 
a B-frame; 

(b) if said current frame is a P-frame, obtaining a code 50 
from a table of codes based on said current frame being 
a P-frame; and 

(c) if said current frame is a B-frame, obtaining a code 
from a table of codes based on said current frame being 
a B-frame. 55 

13. The method of estimating frame-to-frame motion of a 
block of pels in a video frame of claim 12 wherein said step 
of obtaining a code from a table of codes based on said 
current frame being a B-frame comprises the step of obtain- 
ing a code from a table of codes based on the distance from 60 
said current frame to said reference frame. 

14. The method of estimating frame-to-frame motion of a 
block of pels in a video frame of claim 12 wherein said step 
of obtaining a code from a table of codes based on said 
current frame being a B-frame comprises the step of obtain- 65 
ing a code from a table of codes based on whether said 
current frame precedes or follows said reference frame. 
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15. The method of estimating frame-to-frame motion of a 
block of pels in a video frame of claim 14 wherein said step 
of obtaining a code from a table of codes based on said 
current frame being a B-frame further comprises the step of 
obtaining a code from a table of codes based on the distance 
from said current frame to said reference frame. 

16. In a digital video data compression apparatus, a 
motion estimation apparatus for estimating the frame-to- 
frame motion of a block of pels in a video frame comprising: 

(a) means for filtering and decimating a block located at 
a position in a current frame for which motion is to be 
estimated thereby establishing a hierarchical search 
block; 

(b) means for forming a search area in a reference frame, 
the search area surrounding the position of the block in 
the current frame; 

(c) means for filtering and decimating the search area in 
the reference frame thereby establishing a hierarchical 
search area; 

(d) means for successively comparing the hierarchical 
search block with various blocks in the hierarchical 
search area; 

(e) means for identifying a specific block within the 
hierarchical search area which best matches the hier- 
archical search block; 

(f) means for forming an augmented block within the 
search area in the reference frame which surrounds a 
region in the search area that corresponds to the region 
within the hierarchical search area which best matches 
the hierarchical search block; 

(g) means for comparing pels in the block in the current 
frame with pels in various successive blocks in the 
augmented block; 

(h) means for establishing a motion vector for the block 
in the current frame equal to a difference between a 
position of that block in the current frame and a 
position in the augmented block in the reference frame 
which best matches the block in said current frame. 

17. In a digital video data compression apparatus, a 
motion estimation apparatus for estimating the frame-to- 
frame motion of a block of pels In a video frame comprising: 

(a) means for identifying a reference frame; 

(b) means for identifying a block located at a position in 
a current frame for which motion is to be estimated; 

(c) means for determining a size for a search area in the 
reference frame; 

(d) means for determining if the size of the search area 
exceeds an established threshold for regulating estima- 
tion of frame-to-frame motion; 

(e) means for filtering and decimating a block in the 
current frame thereby establishing a hierarchical search 
block; 

(f) means for forming a search area in the reference frame, 
the search area surrounding the position of the block in 
the current frame; 

(g) means for filtering and decimating the search area in 
the reference frame thereby establishing a hierarchical 
search area; 

(h) means for successively comparing the hierarchical 
search block with various blocks in the hierarchical 
search area; 

(i) means for identifying a specific block within the 
hierarchical search area which best matches the hier- 
archical search block; 
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(j) means for forming an augmented block within the 
search area in the reference frame which surrounds a 
region in the search area that corresponds to the region 
within the hierarchical search area which best matches 
the hierarchical search block; 

(k) means for comparing pels in the block in the current 
frame with pels in various successive blocks in the 
augmented block; 

(1) means for establishing a motion vector for the block in 
the current frame equal to a difference between a 
position of that block in the current frame and a 
position in the augmented block in the reference frame 
which best matches the block in said current frame; 
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(m) means for forming a search window in the reference 
frame, with said search window in the reference frame 
surrounding the position of the block in the current 
frame; 

(n) means for comparing pels in the block in the current 
frame with pels in various successive blocks in the 
search window in the reference frame; and 

(o) means for establishing a motion vector for the block 
in the current frame equal to a difference between a 
position of that block in the current frame and a 
position in the search window in the reference frame 
which best matches the block in the current frame. 
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